overview
This session will focus on interaction as a means of storytelling. Whether by explicitly communicating concepts, by evoking emotions or by encouraging certain behaviors, interactions are an integral part to how one exists in the world.
After looking at how interactions can be broadly categorized, we'll discuss how they augment existing spaces and objects, and how they reinforce our fictions.
summary
- intro
- intro
- homework review
- homework review
- prelude
- prelude
- objects
- objects
- kinds of interaction
- kinds of interaction
- augmented realities
- augmented realities
- poetics of interaction
- poetics of interaction
- outro
- outro
intro
responsive spaces
welcome!
plan for the day
when things react, what do they say?
-> homework review
-> prelude
-> types of interaction
-> augmented reality
-> poetics of interaction
homework review
expressive architecture
what ideas/emotions did you choose?
how did you approach it in building the space?
the pattern language
from christopher alexander to software architecture and music.
a pattern is a unit which can repeat and recombine itself in order to form more complex units. they can be considered building blocks of the final whole.
Identifying the patterns in your world can help you figure out what are its most important formal parts. For instance, do you focus on a recurring sentence? On a recurring image? On a recurring lighting?
Once you've identified this pattern, you can modify it in order to highlight particular elements or moments of your experience. Patterns gain their strength through repetition, and inserting something which stands out from such a repetition is one way to grab the attention of your audience and create a dramatic moment.
prelude
interactions in a museum
which interaction? which museum?
good interaction
- having a follower in the museum, having someone watch and follow
- people being more interesting than the artworks, being dynamic, random
- shadow wall, spinning table, watching people interact with the work
- brings something additional, to make you learn something, enjoy
- three level tall, 360 projected/painted panorama, evolves over time
- live media, multi-sensorial
- it interacts with itself
- additional senses mobilized rather than just vision
- using the unexpected
bad interaction
- limited in scope and expectations
- not coherent / no clear purpose
acting and interacting
what's the difference?
interaction: choice & chance, result is one of many possibilities, different perspectives, responsiveness in which the choice has been recognized/has been taken into account
action: cause-effect, deterministically responsive
The difference is between a monologue and a dialogue. In terms of acting, the action doesn't have to come from the human, but can also emanate from the non-human—e.g. imposing behavior such as elevators, motion-activated doors, etc.
Sometimes, museums do involve actions (reading through documents, moving from one piece to the other), but less often do they allow for interactions.
landmarks in your world
based on the concept of landmarks (socio-historical element), paths (direction and speed of movement), districts (coherent ensembles).
which landmark(s) could your your world have?
which path(s) would it have?
which district(s) would it have?
erin/greg/alexis/reeca
field trips, public spaces as teleporting devices
- landmark: arches
- district: cultural spaces (wash sq, arc de triomphe, mumbai, cdmx, etc.)
- paths: undefined?
paul & jack:
- metaverse
- landmarks: technological (are there even any?)
- districts: distributed cities: based on activities (functional worlds)
- agglomerate institutions (e.g. one museum)
vince, sammy, beth:
art gallery
- landmark: statue for empowerment/front desk/model over an archway/atrium/wall painting/"giving someone their flowers"/large tree, for growth (power over vs. power with)
- paths: on-rails (order is randomized) (ref. ikea)
- start: spawn? walk-in?
- exit: button for teleportation?
- districts: debriefing/credits/dance/sculpture/interactive art
results:
yuchen & shaun:
- district: abandoned buildings
- district: housing buildings
- district: bubbles
- landmark: mountain which creates the bubbles
- paths: traced by previous people
frost & jesse:
- landmark: ruin of st paul (macau)
- landmark: large electro-magnetic space launcher + space colony (beacons, sounds)
- landmark: smoke coming from underground (smoke)
- landmark: colony experience room
- landmark: giant planet
- path: red carpet style (ceremony, religious)
hiren & amanda:
- from surface area to depths (going through human history as you go deeper -> districts?)
- landmark: toxic lake
- landmark: open mine/circular ruin
- landmark: sparse residential area
- at the end of the depth, mining disaster happens (last district)
- path: touring path is the one the miners took, fixed route, you choose the speed
- change of miner within depths
objects
objects vs. artefacts
an object is something which is distant from us
an artefact/thing is something we use frequently/intimately
proofs and testimonies
objects act as proof of a collective narrative
artefacts/things act as proof of an individual narrative
meaning in objects
objects have a web of meanings attached to it.
one framework for thinking about it is (paul dugay and stuart hall, doing cultural studies):
- production
- mediation
- consumption
iPhone
produced?
- "Designed by Apple in California. Assembled in China"
- unibody, all-in-one solution
mediation?
- creative tool (shot on iPhone, unleash your inner artist)
- WWDC, yearly cycles, big presentations
- newness and repetition
- apple ecosystem/privacy
- genius bar
- "i'm a mac i'm a pc"
- android vs. ios differs in different regions of the globe
consumption?
- apple product as class marker
- creative (e.g. filmmaking) vs. cameras
- connecting to friends/relatives
- killing time
- app store vs. screen time
- move to mobile-first
- flashlight/measure app
- right to repair
iPhone
produced?
- precise/machining/unibody
- environmental friendly / recycled materials
- foxconn factory workers ("designed in california, produced in china")
- electronic waste
presented/mediated?
- simple
- groundbreaking
- minimalism
- appeal to novelty (new iPhone every year)
- apple is the richest company in the world
consumed/used?
- fashion / class marker
- apple ecosystem (home, airplay, icloud)
- communicate (iMessage, apps)
- take pictures ("shot with iPhone") -> selfies, filmmaking
objects and humans
objects can be classified. based on the categories we choose to put objects, we represent a particular worldview.
means of communication (face to face voice, writing, letter, emoji, sms, email, phone call, smoke signals, brain implant).
- moment in time that they appeared (implicit: progress)
- intimacy
- popularity
- accessibility
- digital vs. analogue
- lasting (ephemeral vs. stored)
objects can be classifiers. based on which object associated with which human, we put a label on those humans.
- iphone vs. android vs. simple phone
- modes of transportation: sportscar, scooter, bike, subway
- clothing: elaborate, non-practical clothing, work clothes w/ stains, uniforms (schools, hospitals, institutions)
kinds of interaction
four kinds
let's focus on these kinds of interactivity and see how they can contribute to world-building.
- implicit
- explicit
- active
- passive
- internal
- external
we've traditionally looked at how interactions should be designed from a usability and affordability standpoint: what makes them obvious, satisfying, etc. we'll start from this point and inquire into what interactions can mean, and suggest in and of themselves.
implicit
the ones we take the most for granted (walking, looking, pressing buttons)
breaking the fourth wall, bringing you into the story (audience involvement), brings insights directly
Working with implicit interactions requires a strong readjustment from the user but, if well done, brings them deeper into your world.
These are often the interactions which are self-reflecting, and break the fourth wall. The picture here is from Metal Gear Solid, in which the player has to battle Psycho Mantis, a boss with psychic abilities. Since Psycho Mantis can read the player's mind (i.e. its input), the way to beat the section is to switch the controller from the slot 1 plug to the slot 2 plug, thus extending outside of the fictional game boundaries.
explicit
requires deliberate action from the user through affordances
and can be highly expressive, when relevant
or completely distracting when off the mark
Explicit interactions are the more classical kinds. When well designed (i.e. providing an interesting, well-balanced choice), they further the engagement of the user, step by step.
When done with the bare minimum of interest, they are just enough to sustain curiosity.
requires:
- affordance
- action
- status (information communication)
active
one-time interaction, with a clear beginning, middle, end, but not always with a clear affordance
there is a before and an after, and usually focus on verifying progress or creating parentheses.
The game here is called Passage, and is a simulation of going through life, in the time span of a video game. Each step is a distinct progression towards the end/the death and can only be avoided by refusing to interact, giving a sense of imposition but also a sense of urgency to the visitor.
passive
these interactions create longer narrative arcs, where the line between interacting and not interacting becomes blurry,
creates an overall environment to be in, becomes a part of what the user does (and is).
The first image is from the Radiophonic Spaces exhibition, in which the visitor walks around the space and, by getting closer or further from the black cables, can tune in to older radio broadcasts. As long as the headphones is set, the visit is one long ambient interaction.
The second image is from a piece called ACCESS
augmenting
how does interacting improve our understanding of a system/environment?
- open: establishing relationships between elements of the world (constructivism)
- close: enacting a given behaviour
it helps broaden scope, makes it more tangible (reading code vs. using code)
reinforces the engagement
develop and test out hypotheses and theories about cause and effect
figure out the limits
highlighting hierarchies and connections and relationships
expose existing relationships
- spotlight: relationship to the self, to others, to being put on a pedestal
- compare things
establish new relationships
add another emotion level (teargas in a museum), involving different senses: persuading and convincing
constructivism, do-it-yourself, non-linear knowledge
showing the invisible
augmented realities
a new layer
constant interaction between the physical and digital is a new layer in the worlds.
while this distinction might be hard to make in a fully-digital world, it can still be a useful one.
Augmentation is the connection between the immediately visible and the temporarily invisible. For instance, when looking at a car, one doesn't immediately know how much carbon dioxide it emits. Augmenting the car could mean displaying, through digital means, the carbon footprint of such car. Similarly, for spaces, one could overlay digitally all the different histories of a given location (one use cases of this is memorials).
The picture here is Listening Post, an installation in the NY Times office lobby, which streams in real-time headlines from across the world, making a tangible connection between data streams and brick and mortar.
augmenting the world
because augmentation is ad hoc, it moves away from being something that everyone will see.
By adding more versions and more filters, you might be able to provide even more versions of your world, but risk getting further and further away from a common ground.
While this is particularly true of augmentations of the physical world, this also applies to digital worlds (e.g. filter bubbles as they develop on social media, where none of us ever see the same thing, except the overall structure of the page).
interacting with information
it's about how easy information access is:
is it secret?
publicly available?
slammed on our screens?
Covid health signs vs. graffitis on a wall
TV broadcasts vs. secret diaries
always on vs. always off.
information interaction is not just about good looking interfaces, but about accessible information.
the things that you make easily accessible are the things that will weigh more in your world, they will take up more space and eclipse information that might be buried down the last page. on the other hand, information that is being searched for actively by a visitor is information that will carry deeper signification to them.
again, it's a tradeoff.
diegetic and non-diegetic
diegetic: belongs to the world of the game
non-diegetic: does not belong to the world of the game
poetics of interaction
poetics of interactive space
interactions have a character, and can therefore clash or reinforce the fictions and the artistic direction of your world.
meaningful interaction is when you don't assume that the interactor is stupid.
Participation as an activity is not, however, dependent on technology; it is rather an historical elaboration and transmutation of dialogic modes of encounter, the archetype of which is face-to-face conversation. Now, however, one "interfaces" or communicates by means of a computermediated simulation. To interact is a kind of doing that entails purposiveness, conclusiveness, and agency—qualities that, namely, point to a subject. ... The utopian claims for interactivity as a liberatory and nonhierarchical praxis are based on the capacity to accommodate multiple and nonlinear links between elements in narration and the potentially more egalitarian or dialogic relation between artists and their audiences.
Margaret Morse, Poetics of Interactivity, 2003.
expressive motion
what would be the main interaction of your world?
how is it triggered?
what gets affected?
for how long?
why?
ideas
outro
conclusion
interactions can take place implicitly or explicitly, over space, or over time
they are part of your world making, of its coherence.
homework
use triggers and colliders to create immediate interactions, affecting your world through light.
think of how you make it explicit or implicit, dramatic/poetic or mundane/subtle (everyday, common), understandable/unexpected.
think of how you reconfigure space and redirect attention through the use of light, scale of objects.